A Physical Sampling Metric for Image-based Computer Graphics

Computer models of the real world often use images of the environment to capture realistic visual complexity. Image-based modeling techniques permit the creation of geometric models with a high level of visual detail from photographs. These models are textured by resampling these images of the scene; we call this process image-based texturing. The problem with traditional image-based texturing is the poor quality of the extracted textures, which are often blurred or stretched due to sampling problems. Furthermore, the extent of this degradation varies across the scene, due to differences in the pose and position of the camera relative to each object in each image. This thesis makes two contributions to image-based computer graphics. First, it introduces a physically-based metric of sampling quality, based on the Jacobian matrix of the imaging transform, which captures the interaction of the imaging system with the imaged environment. This metric provides a direct, physical measure of the quality of resampled textures, and suggests a physical interpretation of the multi-resolution image representations widely used in texture synthesis. The second contribution, which builds on this insight, is a novel use of the metric for extending current texture synthesis methods to image-based texturing processes. Use of the sampling metric enables detail synthesis – the insertion of high spatial frequency detail into regions of an image-based model’s textures where the imaging process captures only low frequency texture data. Given a small set of input images and a geometric model of the scene, this technique allows the creation of uniform, high-resolution textures. Our synthesis approach relieves the user of the burden of collecting large numbers of images and increases the quality of user-driven imagebased modeling systems. The research described in this thesis allows both the quantification of sampling effects in image-based computer graphics systems, as well as the correction of degradation in image-based textures. The sampling metric introduced in this thesis has usefulness far outside the image-based texturing application demonstrated here. Such a metric will have a potential impact in the fields of vision-based geometric reconstruction, material measurement, image-based rendering, and geometric level-of-detail management. The goal of this thesis is merely to introduce the metric and validate its usefulness for one critical application.